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Abstract 

Indonesia's involvement in the Programme for International Student 
Assessment (PISA) is one attempt to see how far the development of 
educational programs in our country compared to other countries in the 
world. PISA results show that Indonesia is still at the lower level. This 
means that the ability of Indonesian students in solving problems that 
require the ability to review, giving reasons and communicating 
effectively, and solve and interpret problems in various situations is 
still lacking. This may be due to government policy in the presence of 
the National Examination (UN) in which the spread of the UN’s 
questions are still at the lower levels of cognitive aspects that are not in 
line with government regulations on curriculum which suggests that 
the fulfillment of cognitive aspects as one of the important aspects of 
education. To that end, researcher conducted a study that aims to 
produce valid and practical the sixth level of PISA-like mathematics 
problems for middle school students. This study is the development 
research formative evaluation type. The research subjects are ninth 
grade students SMP Negeri 1 Palembang. Data collection techniques 
used are walk through, documentation, interviews, and tests. From the 
analysis it can be concluded that this research has resulted a product 
the sixth level of PISA-like mathematics problems. At the stage of 
expert review, an expert and two colleagues evaluated the problems 
from different aspects. Trying out at one-to-one and small group was 
performed on students with different mathematical abilities. Then at 
the field test stage, 26 students in one class answered the questions that 
were developed. 

Keywords: development research, PISA questions, development the 
sixth level of PISA-like mathematics problems 

Abstrak 

Keterlibatan Indonesia dalam Program International for Student 
Assessment (PISA) adalah salah satu upaya untuk melihat sejauh mana 
perkembangan program pendidikan di negara kita dibandingkan 
dengan negara-negara lain di dunia. Hasil PISA menunjukkan bahwa 
Indonesia masih berada pada level bawah. Ini berarti bahwa 
kemampuan siswa Indonesia dalam menyelesaikan soal-soal yang 
menuntut kemampuan untuk menelaah, memberi alasan dan 
mengomunikasikannya secara efektif, serta memecahkan dan 
menginterpretasikan permasalahan dalam berbagai situasi masih sangat 
kurang. Hal ini mungkin disebabkan kebijakan pemerintah dengan 
adanya Ujian Nasional (UN) di mana penyebaran soal UN masih 
berada pada level bawah dari aspek kognitif yang tidak sejalan dengan 
peraturan pemerintah mengenai KTSP yang mengisyaratkan agar 
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terpenuhinya aspek kognitif sebagai salah satu aspek penting dalam 
pendidikan. Untuk itu, peneliti melakukan penelitian yang bertujuan 
untuk menghasilkan soal matematika model PISA level 6 untuk siswa 
sekolah menengah pertama yang valid dan praktis. Penelitian ini 
adalah penelitian pengembangan ( development research ) tipe 
formative evaluation. Subjek penelitian adalah siswa kelas IX SMP 
Negeri 1 Palembang. Teknik pengumpulan data yang digunakan adalah 
walk through, dokumentasi, wawancara, dan tes. Dari hasil analisis 
dapat disimpulkan bahwa penelitian ini telah menghasilkan suatu 
produk soal matematika model PISA level 6. Pada tahap expert review, 
tiga orang pakar dan teman sejawat mengevaluasi dari aspek yang 
berbeda. Uji coba soal pada tahap one-to-one dan small group 
dilakukan pada siswa dengan kemampuan matematika yang berbeda. 
Pada tahap field test, 26 orang siswa dalam satu kelas menjawab soal- 
soal yang dikembangkan. 

Kata Kunci: penelitian pengembangan, soal PISA, pengembangan 
soal matematika model PISA level 6 


Introduction 

Development of science and technology is faster and faster, especially on information 
and communication scope so that human resources that are reliable and able to 
compete globally are necessary. Indonesian students should be able to compete with 
other students from various countries in the face of the current era of globalization. 
Various tests conducted internationally can be used as a benchmark to determine the 
extent to which Indonesian students competing in the globalization era. The results of 
these tests are often used as a basis for program development and improvement of 
education quality. Indonesia's involvement in the Programme for International Student 
Assessment (PISA) is one attempt to see how far the development of educational 
programs in our country compared to other countries in the world. 

PISA results show that Indonesia is still at the lower level. Of the 65 participating 
countries in PISA 2009, Indonesia occupied the position 61 st for PISA mathematics. 
This means that the ability of Indonesian students in solving problems that require the 
ability to review, giving reasons and communicating effectively, and solve and 
interpret problems in various situations is still lacking. Based on the results of PISA 
mathematics in 2009 (Stacey, 2010), only 0.1% of Indonesian students are able to 
solve problems for the two top level in PISA. 

This may be due to government policy in the presence of the National Examination 
(UN). Based on research results Sampoerna Foundation (Yunengsih, Widiatmika & 
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Candrasari, 2008) about the deployment of UN remains at the level of cognitive 
aspects memorize, perform the procedure, and demonstrate understanding. As for the 
other two aspects namely conjecture/generalize/prove and solve non-routine problems 
not at all touched by the UN in the subject matter of mathematics. Yet these two 
aspects occupy the highest levels of cognitive aspects. This is inconsistent with 
government regulations on curriculum which suggests that the fulfillment of the 
cognitive aspects as one of the important aspects of education. 

Evy Yosita Silva (2011) have developed mathematical questions PISA-model on the 
content uncertainty for ninth grade secondary school students that are grouped based 
on clusters competencies in PISA. However, this study did not categorize the 
questions that have been developed into the PISA levels. In addition, Annisah (2011) 
also developed mathematical questions PISA-model on content quantity level 2 
through 6 for measuring mathematical reasoning ability for middle school students. 
The results of this study include the inability of students in identifying problems and 
looking at the matter of the relevant mathematical way to solve the given problem. 
Aiming to produce the sixth level of PISA-like mathematics problems that are valid 
and practical, the research question of this study is how to develop valid and practical 
mathematics questions PISA-mode level 6 for secondary school students. The results 
of this research are expected to be useful for students, so they accustomed to use their 
knowledge in everyday life by answering mathematical questions PISA-model, for 
mathematics teacher, to use the questions as an alternative in the improvement of 
learning evaluation that can be used to train students in applying mathematics skills 
and knowledge in everyday life. 

Theoritical Framework 

1. Programme for International Student Assessment (PISA) 

PISA is a collaborative effort on the part of the member countries of the OECD to 
measure how well students at age 15, and therefore approaching the end of 
compulsory schooling, are prepared to meet the challenges of today’s societies. PISA 
assessment takes a broad approach to assessing knowledge and skills that reflect the 
current changes in curricula, moving beyond the school based approach towards the 
use of knowledge in everyday tasks and challenges. PISA combines the assessment of 
domain-specific areas such as reading, mathematical, and scientific literacy with 
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important cross-curricular areas, also a priority among OECD countries (OECD, 
2003). PISA assessment has been conducted for four periods, namely 2000, 2003, 
2006, and 2009. In the 2000, PISA study focused on reading ability, while two other 
aspects as a companion. In 2003, mathematical literacy became the main focus, then 
science literacy in 2006 and in 2009, the main focus shifting from science literacy 
towards reading literacy. 

The mathematical literacy definition for PISA is (OECD, 2009a): 

Mathematical literacy is an individual’s capacity to identify and 
understand the role that mathematics plays in the world, to make 
well-founded judgements and to use and engage with mathematics in 
ways that meet the needs of that individual’s life as a constructive, 
concerned and reflective citizen. 

Thus knowledge and understanding of mathematical concepts is important, but the 
ability to turn on mathematical literacy to solve problems encountered in daily life is 
more important. Therefore, the questions given in PISA are presented mostly in the 
context of real-world situations that can be felt the benefits of mathematics to solve 
problems of daily life. The purpose of PISA is to measure the level of students' skills 
in using mathematical knowledge and skills to handle everyday problems. 

2. Mathematical Content - The four “overaching ideas” - in PISA 
Mathematical concepts, structures, and ideas have been invented as tools to organize 
he phenomena of the natural, social and mental world. Since the goal of PISA is to 
assess students’ capacity to solve real problems, the strategy has been to define the 
range of content that will be assessed using a phenomenological approach to 
describing the mathematical concepts, structures or ideas. This means describing 
content in relation to the phenomena and to kinds of problems for which it was 
created. This approach ensures a focus in the assessment that is consistent with the 
domain definition, yet covers a range of content that includes what is typically found 
in other mathematics assessments and in national mathematics curricula. 

The following list of overaching ideas is used in PISA to meet the requirements of 
historical development, coverage of the domain, and reflection of the major threads of 
school curriculum: quantity, space and shape, change and relationship, and 
uncertainty. With these four, mathematical content is organized into a sufficient 
number of areas to ensure a spread of items across the curriculum, but at the same 



13 

Developing The sixth level of PISA-Iike mathematics problems for Secondary School 
Students 

time a number small enough to avoid a too fine division that would work against a 
focus on problems and concepts that make sense and can be encountered within and 
across a multitude of different situations. 

3. Situations or Context in PISA 

An important aspect of mathematical literacy is engangement with mathematics: using 
and doing mathematics in a variety of situations. It has been recognized that in dealing 
with issues that lend themselves to a mathematical treatment, the choice of 
mathematical methods and representations is often dependent on the situation in 
which the problems are presented. The situation is the part of the student’s world in 
which the tasks are placed. It is located at a certain distance from the students. For 
OECD/PISA, the closest situation is the student’s personal life; next is school life, 
work life, and leisure, followed by local community and society as encountered in 
daily life. Furthest away are scientific situations. Four situation- types are used for 
problems to be solved: personal, educational/occupational, public, and scientific. The 
context of an item is its specific setting within a situation. It includes all the detailed 
elements used to formulate the problem. 

4. The PISA Scale and Difficulty 

To summarise data from responses to the PISA test instrument, a six-level described 
performance scale is created. The scale created statistically, using an item response 
modeling approach to scaling ordered outcome data. The overall scale is used to 
describe the nature of performance by classifying the students performances of 
different countries in terms of the five described performance levels, and thus provide 
a frame of reference for international comparisons. 

At proficiency level 6, students can conceptualise, generalize, and utilize information 
based on their investigations and modeling of complex problem situations. They can 
link different information sources and representations and flexibly translate among 
them. Students at this level are capable of advanced mathematical thinking and 
reasoning. These students can apply this insight and understandings along with a 
mastery of symbolic and formal mathematical operations and relationships to develop 
new approaches and strategies for attacking novel situations. Students at this level can 
formulate and precisely communicate their actions and reflections regarding their 
findings, interpretations, arguments, and the approprieteness of these to the original 
situations. 
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5. Mathematical Literacy and Mathematics Learning Goals 

Based on the definition of mathematical literacy that is mentioned in the previous 
section, mathematical literacy is concerned with student's ability to formulate, 
implement, and interpret mathematics in various contexts, including the ability to 
reason mathematically and use the concepts, procedures, and facts to illustrate, 
explain or predict phenomena. Mathematical literacy helps one to understand the role 
or usefulness of mathematics in everyday life as well uses it to make the right 
decisions as citizens. 

In line with this, Permend ik nas No. 22, 2006 about content standards of mathematics 
subject for basic education scope states that mathematics courses intended that 
learners have the following capabilities (Wardhani, 2008). 

1. Understanding mathematical concepts, explain the link between concepts or 
algorithms flexibly, accurately, efficiently, and appropriately, in solving problems. 

2. Using the reasoning on the pattern and nature, performing mathematical 
manipulation in making generalizations, compiling evidence, or explaining ideas 
and mathematical statements. 

3. Solving problems that include the ability to understand the problem, designing a 
mathematical model, completing model and interpreting the obtained solution. 

4. Communicating ideas with symbols, tables, diagrams, or other media to clarify the 
situation or problem. 

5. Having respect for the usefulness of mathematics in life, have a curiosity, 
attention, and interest in studying mathematics, and a tenacious attitude and 
confidence in problem solving. 

If mathematical literacy is compared with the mathematics learning goals then there is 
suitability and understanding. The learning golas to be used in Permendiknas is 
mathematical literacy. Note that, the goals for mathematics subject according to 
content standards are essentially an ability also known as mathematical literacy. 

Method 

The research was conducted in odd semesters of the school year 2011/2012 in ninth 
grade SMP Negeri 1 Palembang. The method in this study is development research 
formative evaluation type (Tessmer, 1993) which includes expert review and one-to- 
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one, small group, and field test. At expert review, the designed products were 
observed, assessed and evaluated by experts. Experts were examining content, 
construct, and the language of each prototype. The suggestions of the experts were 
used to revise the questions that have been developed. In one-to-one, the researcher 
tested the questions to students as a tester. Students' answers and comments used to 
revise the questions that have been made. 

The results of revision of the experts and the difficulties experienced by students 
while testing the first prototype were used as a basis for revising the prototype and 
then the result is called the second prototype tested in small groups. At this stage, six 
students grade nine were asked to solve the problems that have been designed and 
were also asked their comments on that question through interviews. Suggestions and 
comments and the test results on a second prototype used as a basis for revising the 
design of the second prototype. The results of revisions were tried out to the subjects 
in this research as a field test. The sequence of formative evaluation is as follows. 



Figure 1. General sequence of formative evaluation types (Tessmer, 1993) 


Data collection techniques used in this study is walk through to find out the 
qualitative validity of the content, construct, language; documentation to determine 
the practicality of a matter; interviews to determine students 'suggestions and 
comments, and tests to determine students' answers to the questions that were 
developed. 

Result and Analysis 

In the early stages, some preparation is done which includes analysis of students to 
determine the level of mathematical ability of students in choosing a subject of 
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research, analysis the curriculum by indentifying mathematics learning material for 
secondary school student, analyzing PISA questions by reviewing several sources of 
PISA to identify the characteristics of PISA questions level 6, then designing the sixth 
level of PISA-like mathematics problems. Furthermore, based on the development 
steps described earlier, the stages are as follows. 

1. Expert Review 

At this stage the qualitative validity were consulted and examined based on the 
content, construct, and language to the professors and colleagues who have experience 
in mathematics education as a validator, the validator is: Prof. Dr. Sutarto Hadi, M. Si, 

M. Sc, professors and lecturers of mathematics education courses Lambung 
Mangkurat University, Banjarmasin to review the mathematical concepts and contexts 
used in the questions that are already developed in accordance with the characteristics 
of PISA questions level 6. Nurjannah, S. Pd, a Secondary School mathematics teacher 
who reviewed the correspondence between problems developed with the level of 
secondaryh school students in grade nine also with Indonesian curriculum, interest 
students to solve, and interest mathematics teachers to use these problems in teaching 
and learning. Rully Charitas Prahmana Indra, writer and a student of Sriwijaya 
University Graduate Program as an editor of the writing and the use of sentences in 
the questions that were developed to improve the clarity. 

2. One-to-one 

At one-to-one, in which the questions developed were tested to three students with 
different mathematical abilities, namely high, medium, and low. The procedure was 
same for each student, although it was performed at different times. The researcher 
met with each student individually and involved them in an informal conversation. 
Evaluation of one-to-one focusing on clarity, ease of use, and effectiveness of the 
questions that were developed, as well as the interest of students to such questions. 

Here are the comments of students on the stage of one-to-one. 

Tabel 1. Students’ comments in one-to-one evaluation 

_ Students’ name _ Comments _ 

Farah Ayodya Swari Not accustomed to work on problems l ik e this, 

(high mathematical ability Usually work on the problems using the formula 
students) Do not l ik e doing it because it is difficult to read the long 

questions 

Fakhriah Febriana Fitria Accustomed to work on the problems l ik e this 

(medium mathematical Actually, a little confused how to solve it but I enjoy it. 
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ability students) Must be more carefully in order to answer the questions. 

Deandra S. Like to do the problem like this because it is good for our brain, 

(low mathematical ability The words and sentences on the matter, most obviously 
students) Difficult to provide explanations for the answers given. 


After the expert review phase and one-to-one, the questions were revised in order to 
obtain prorotype II tested in small group. 

3. Small group 

At this stage, the researcher was only as an observer in which the questions were 
tested to six ninth grade students SMPN 1 Palembang. Students are required to solve 
all the questions provided and then provide suggestions/comments on the matter that 
has been done. 


Tabel 2. Students’ comments in small group evaluation 


Students’ name _ 

Tk limah dan Dasi Edimas • 
(high mathematical ability 
students) • 


Fadel Muhammad and • 
Fatya Rimandita Hero • 

(medium mathematical 
ability students) • 

Pingkan Taradhita and • 
Muhammad Adil Wira S. 
(medium mathematical • 
ability students) 


4. Field test 


_ Comments _ 

The problem is very difficult and the words make me 
dizzy and confused. 

The problems given are quite difficult, not only have 
to calculate, but also must use logic correctly to find 
the answer. 

Actually, this problem is not too difficult. 

Although I had found the results, but as if hard to 
articulate explanation. 

The questions are rather difficult because the 
questions are difficult to understand. 

The questions are easy and the answers are also easy 
to obtain. 

The questions are quite difficult but I can answer 
them. 

Difficult to explain how to find the answer. _ 


Field tests conducted on November 28 th , 2011 in class IX 2 SMPN 1 Palembang with 


the number of students were 26 people consisting of 12 male students and 14 female 
students. Twelve questions were given to students and worked for 2 hours of lessons 
(90 minutes). 


The sixth level of PISA-like mathematics problems were categorized qualitatively 
valid and practical. Validity is reflected from the results of the validator, the validator 
states where almost all items based on the content (in accordance with the 


characteristics PISA, mathematical ability level indicator 6 PISA), construct (develop 
mathematical literacy skills, rich with the concept, accordance with secondary school 
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students grade nine, inviting further development of the concept), and language (in 
accordance with the EYD, the questions not beating around bush, the quesions do not 
contain a double interpretation, the limits of the questions and answers is clear). 

The revision based on comments/suggestions of students on one-to-one and small 
group evaluation showed that the questions developed are practical in which all 
students can use the device well. The questions developed in accordance with the 
student's train of thought, the contexts given are familiar to the students, easy to read, 
and not cause a diverse interpretation. 

The following is an explanation about the the sixth level of PISA-like mathematics 
problems as well as some students' answers on the field test: 

MOTIF BATIK 



n = 1 n = 2 n = 3 


Sumber: http://kamusjawa.com/wp-content/uploads/2011/Q8/batik kawungl.jpg 
Untuk kain dengan luas 6 cm”, dilukis 1 buah bunga putih yang terdiri dari 4 
kelopak, kemudian untuk kain seluas 12 cm” dilukis 4 buah bunga putih, dan 
seterusnya. Jika n adalah jumlah baris dari bunga putih maka 
Jumlah bunga putih = n dan luas kain = 6 n 

Andaikan kain yang digunakan semakin luas dengan motif bunga putih yang sama 
dengan pola sebelumnya, yang manakah yang meningkat lebih cepat: jumlah bunga 
putih atau luas kain? 

Buktikan dan jelaskan jawabanmu! 
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Content : Quantity 

Konteks : Occupation 

Skoring: 

Full credit: 

Correct answers (the number of white flowers) equipped with a valid explanation, for 
example: 

■ The number of flowers = nx n and the fabric’s wide = n x 6. both formulas have a 
factor of n. but the number of flowers has n that also increased while fabric’s wide 
has 6 that remain constant. So that the number of flowers will increase more 
rapidly; 

■ The number of flowers will increase more rapidly because the number will be 
squared compared with multiplied by 6; or 

■ The number of flowers is a quadratic number, fabric’s wide is linear. Thus, the 
number of flowers will rise faster. 

Partial credit 

Correct answers (the number of white flowers) equipped with an explanation based on 

examples, for example: 

■ Using table, 



n= 1 

n = 2 

n = 3 

n = 4 

n = 5 

n = 6 

n = 7 

Fabric’s wide 

6 

12 

18 

24 

30 

36 

42 

The number of 
white flowers 

1 

4 

9 

16 

25 

36 

49 


■ The number of white flowers will raise faster if n> 6. 
No credit: wrong answer and missing 


The question is related to the subject of algebra associated with multiplication. This 
relationship is expressed in algebra symbol n. Because every representation of a 
symbol that has the purpose and nature of each, the process of translation becomes 
very important and determines in accordance with the situation and work to be done. 
Context of work on a matter related to batik made by craftsmen. Students' knowledge 
about mathematical concepts is expected to help formulating, conducting the 
classification problem, and solving the problem of employment in general. 

Based on indicators level 6, students should be conceptualized, generalize, and use 
information based on extensive investigation and modeling of fabric’s wide and the 
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number of flowers expressed in n then connect the source of information and different 
representations for fabric’s wide and the number of interest with their respective size 
and flexibly translate the increase in comparable of fabric’s wide and the number of 
flowers where if the number of fabric’s wide increases, the number of flowers will 
also increase. In addition, students need to demonstrate mathematical thinking and 
good reasoning to determine which one is increasing faster and apply this insight 
along with a mastery of symbolic and formal mathematical operations and 
relationships (a quadratic number and a result of multiplying a number by a constant 
number) to develop an approach and a new strategy to determine the extent of fabric’s 
wide or the number of flowers will rise even faster, then formulate and communicate 
with appropriate action and reflection on the discovery, interpretation, argument, and 
the suitability of the discovery, interpretation, and argument with the original 
situation. 

Only 7.7% of students were able to answer this question with a valid explanation 
furnished. They were able conceptualized, generalize, and use information based on 
extensive investigation and modeling of fabric’s wide and the number of flower 
expressed in n then connect it with the equation. They have also demonstrated the 
mathematical thinking and good reasoning to determine the amount of increase due to 
the increased of flower in the quadratic equation (Figure 2). In addition, Mahali 
answered this question correctly but with the explanation just using example (Figure 
3) so that when n> 6 he realized that the number of flower is increasing faster than the 
broad fabric. There were also 7.7% of students answering using examples like this. 
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Figure 2. Shafira’s answer for the first question 
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Figure 3. Mahali’s answer for the first question 


TEOREMA PYTHAGORAS 

Teorema Pythagoras menyatakan bahwa: jumlah kuadrat 
kedua sisi siku-siku pada segitiga siku-siku sama dengan 
panjang kuadrat sisi miringnya, yaitu a 2 + b 2 = c 2 seperti 
gambar di samping. 

Perhatikan Gambar di bawah ini! 



Gambar A 



Gambar C 




Gambar B 

b 


Gambar D 


Soal 11 

Tulislah “Ya” atau “Tidak” untuk setiap gambar yang juga dapat digunakan untuk 
membuktikan teorema Pythagoras. 


Gambar yang dapat digunakan 

Gambar ini dapat digunakan 
untuk membuktikan a 2 + b 2 = c 2 

Gambar A 

Ya / Tidak 

Gambar B 

Ya / Tidak 

Gambar C 

Ya / Tidak 
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Gambar D 


Ya / Tidak 


2 

Kamu harus membuktikan jawabanmu dengan menunjukkan bahwa persamaan a“ + 
b 2 = c 2 dapat ditemukan pada Gambar A, B, C, dan D di atas! 


Content : Space and shape 

Context : Science 

Scoring 
Full credit: 

Correct answers (in order: Yes, Yes, Yes, No) with valid proof, namely: 
Figure A is a square with side length a + b in which there were four congruent right- 
angled triangle the square with side c then the area of Figure A is 

(a + b) 2 = 4 x (Vi xaxb) + c 2 «a 2 + 2ab + b 2 = 2ab + c 2 <=^> a 2 + b 2 = c 2 
Figure B is a right-angled trapezoid with a length of the parallel sides are a and b in 
which consists of two congruent right-angled triangle and an isosceles right triangle 
with side length c, so the area Figure B is 

Vi (a + b) x (a + b) = 2 x (‘/2 x a x b) +V2 c 2 <=> ¥2 a 2 + Viab + Viab + ¥2 b 2 = 
ab + V2C 2 

<=> V2 a 2 + ¥2 b” = Vkr <=> a” + b” = c 2 

Figure C is a square with side length c within it consists of four congruent right- 
angled triangles, and a square with side length (ab), so the area of Figure C is 
c 2 = 4 x (¥2 x a x b) + (a-b) 2 c 2 = 2ab + a 2 - 2ab + b 2 ^>c 2 = a 2 + b 2 
Figure D is a square with side length a + b in which consists of two square pieces each 
of length a and b sides and four congruent right-angled triangles, so the area of Figure 
D is 

(a + b) 2 = a 2 + b 2 + 4 x (¥2 x a x b) a 2 + 2ab + b 2 = a 2 + b 2 + 2ab 
Partial score: 

Correct answers (in order: Yes, Yes, Yes, No) equipped with a less detailed 
explanation. 

No credit: other responses and missing 


The question is the kind of "quasi-realistic" that is more widely available in 
mathematics than in everyday life. Although not considered a typical, small part about 
this type tested in PISA. Students need the ability to realize that they have to prove 
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whether in Figure A, B, C, and D apply the Pythagorean Theorem. Therefore they 
need to mark the visual information and see pictures of them as a square or trapezoid 
consisting of triangles and use the broad approach to prove it. This question requires 
students to understand the visual information, using argumentation skills, using 
technical knowledge and insight into the geometry, and sustainable use logical 
thinking. 

Based on indicators about level 6, students conceptualized, generalize, and use 
information based on the investigation and modeling of problem situations are 
presented, namely proof of the Pythagorean Theorem using a broad approach. They 
also have to connect the source of information and different representations of the 
images provided and flexibly translate them if could be used to prove the Pythagorean 
theorem. Then the students demonstrate mathematical thinking and reasoning are well 
associated with the broad square, triangle area, large rectangle, and trapezoid area, 
and apply this insight along with a mastery of symbolic and formal mathematical 
operations and relationships to develop a strategy of proof is applicable equation a2 + 
b2 = c2 to images that are available. Formulate and communicate their actions and 
reflections to the right of discovery, interpretation, argument, and the suitability of the 
discovery, interpretation, and argument with the original situation. 

None of the students are able to answer this question correctly. They determine the 
answer just by looking at the pictures provided if there is a right triangle as shown in 
Figure 4. They do not synchronize the pictures to prove the Pythagorean Theorem 
using a broad approach so that they do not develop a strategy of proof is valid. 
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Figure 4. Shafira’2 answer for the second question 
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NILAI PISA 

PISA ( Program for International for Student Assessment) adalah studi literasi yang 
bertujuan untuk meneliti secara berkala tentang kemampuan siswa usia 15 tahun 
(kelas IX SMP dan Kelas X SMA) dalam membaca ( reading litercy ), matematika 
(mathematics literacy), dan sains ( science literacy). 

Seorang mahasiswa menunjukkan grafik berikut dan berkata: 

“Grafik ini menunjukkan bahwa ada peningkatan yang sungguh sangat besar dalam 
rata-rata nilai PISA matematika Indonesia dari tahun 2003 ke tahun 2006.” 

Indonesia: Mean PISA scoresfor 2000-2009 for 
malhematicft, acienneand reading 



Sumber: http://jims-b.org/?page_id=152&download=12 


Apakah menurut kamu, mahasiswa tersebut mengatakan pernyataan yang masuk akal? 
Berikan bukti dan penjelasan untuk mendukung jawabanmu! _ 


Content : Uncertainty 

Context : Public 

Scoring 
Full credit: 

Correct answers (no sense) with the explanation that focuses on the fact that only a 
small graph shown, for example: 

■ All graphs should be shown. 

■ It's not a reasonable interpretation because if the entire graph is shown then it will 
be seen that there is little increase in the value of PISA. 

■ Since the college student shows only a small portion of the graph, if there is the 
whole graph, i.e 0-410, then the value of PISA mathematics Indonesia is not very 
large increase. 
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■ Although the graph shows a large increase, but if the number is concerned, it's not 
a huge increase 

Correct answers (no sense) with the explanation that relates to the comparison and 
percentages. 

■ Since 30 points is not a huge increase when compared with a total of more than 
300; 

■ Since when is calculated based on the percentage, the increase is only about 10%; 

■ Since only about 30 points higher than the previous year was not a huge increase 
when compared with the 360. 

Partial credit: Correct answers (not sense) equipped with a less detailed explanation. 
No credit: Wrong answer without explanation or missing. 

The graph shown in this problem is obtained from the actual graph with little 
oversight. The graph seems to indicate, as the students: "really a very large increase in 
the average value of PISA mathematics Indonesia". Students were asked whether the 
statement is in accordance with existing data. It's important to look through the data 
and graphs, as is often shown in the media in order to function well in the knowledge 
society. This is an important ability in mathematics literacy. Mathematical content 
that exist in this problem is to analyze the graph and interpret data and understand the 
problems associated with data interpretation errors. In this graph, the y-axis cut shows 
that indeed very large increase in the average value of PISA mathematics Indonesia, 
but the real difference between the average value of PISA mathematics Indonesia in 
2003 and 2006 is not really very large (Figure 5). This question requires students to 
understand and mark the graphical representation in a critical manner, making 
assessments and finding the appropriate arguments based on mathematical thinking 
and reasoning (interpretation of the data), using the reasoning in the context of 
comparative statistics and unusual situations, and communicate their reasoning 
process effectively. 
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Figure 5. The graph for the third question 


This question includes PISA level 6 in which students conceptualized, generalize, and 
use information based on the investigation and modeling of the words of a student and 
connect sources of information and graphic representations of the average value of 
PISA Indonesia from 2000 to 2006 and flexibly translate. Then they demonstrate 
mathematical thinking and good reasoning to determine whether the statement of the 
college student in accordance with the chart shown, and apply this insight along with 
a mastery of symbolic and formal mathematical operations and relationships in a 
reading chart to develop approaches and strategies to determine whether the words of 
the incoming students sense. They also formulate and communicate with appropriate 
action and reflection on the discovery, interpretation, argument, and the suitability of 
the discovery, interpretation, and the argument that can support their answers. 

Like the previous problem, there is no student who answered correctly. They did not 
notice that only a small graph shown that an increase in the value of PISA 2003 to the 
year 2006 looks very big. This indicates that they are not yet applying formal 
mathematical operations and relationships in reading graphs to develop approaches 
and strategies in determining whether the student is saying makes sense. 

Based on the above explanation, no students are able to answer correctly the second 
and the third questions. Researchers assume that because that question was included 
level 6 of PISA and students have not been accustomed to work on issues l ik e that, so 
they have not been able to understand the problem well. It is also in line with the 
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results of interviews with 3 ninth graders about the problems. They understand the 
problem after re-described and given instructions that can help them understand it. 
They can answer it and they also have the knowledge necessary to answer such 
questions. 

Conclusion 

This research has resulted the sixth level of PISA-like mathematics problems for 
secondary school students. At the stage of expert review, the questions that were 
developed were evaluated by three experts and colleagues from different aspects 
(mathematical content, compliance with the curriculum and students’ mathematical 
ability, as well as the use of sentences and words on the matter). At one-to-one stage, 
the questions developed tested to three students with different mathematical abilities 
(high, medium, and low) that focuse on clarity, ease of use, and effectiveness, as well 
as the interest of students to solve them. At small group, the questions had been tried 
out to six students who also have different mathematical ability to justify revisions 
made based on expert review and one-to-one and revise again the questions are 
developed. In the field test stage, 26 students in one class answered the questions that 
were developed to determine whether such questions can be used on the actual 
situation. 
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